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[57] ABSTRACT 

Multi-channel color image signals from a digital camera 
having multi -channel image sensors are corrected to account 
for variations in scene illuminant. This is accomplished by 
determining the scene illuminant and determining an opti- 
mum color-correction transformation in response to the 
scene illuminant which transform minimizes color errors 
between an original scene and a reproduced image by 
adjusting three or more parameters. 

29 Claims, 9 Drawing Sheets 
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METHOD AND APPARATUS FOR COLOR- 
CORRECTING MULTI-CHANNEL SIGNALS 
OF A DIGITAL CAMERA 

FIELD OF THE INVENTION 

Ibis invention pertains to the field of digital imaging, and 
more particularly to the color-correction of images obtained 
with digital cameras. 

BACKGROUND OF THE INVENTION 



10 



Digital cameras are becoming increasingly common both 
in the field of still photography, as well as in the field of 
motion imaging as is evidenced by the proliferation of 
digital cameras and video recorders. Digital imaging has the is 
advantage over its counterparts, which utilize conventional 
media such as silver halide film, that the results are instantly 
available for viewing, editing, printing, and other forms of 
utilization. 

A characteristic of color digital imaging devices is that the 20 
digital signals produced by the image sensor will be a 
function of the spectral characteristics of the light used to 
illuminate the scene. For example, if the color processing in 
a camera is designed to give good color reproduction in a 
daylight illumination condition, unacceptable color repro- 25 
duction may be obtained if the camera is used with tungsten, 
flash, or fluorescent illumination. This is due to the fact that 
the response measured by the color sensors is a function of 
the spectral power distribution of the light source as well as 
the spectral reflectance of the objects in the scene, and the 30 
spectral responsivities of the sensors. 

Consider a digital camera having color image sensors that 
are nominally sensitive to the red, green, and blue portions 
of the visible spectrum. The linear color signals produced by 
the camera will be given by 



G-k c fI{k)R(k)S^k)dk 

where R, G, and B are the red, green, and blue color values, 
respectively. 1(1) is the spectral power distribution of the 
illuminant R(X) is the object spectral reflectance. Sj£k), 
S^^), and S^(X) are the spectral responsivities of the red, 
. green, and blue sensors, respectively. The constants k^, k^, 
and kj, are channel dependent gain factors. Typically, these 
gain values are chosen so that equal color signal levels 
(R=G=B) are obtained when a neutral object is photo- 
graphed. 

Generally, the RGB color signals produced by the color 
sensors are not appropriate for display on any given color 
image display device. Examples of typical display devices 
include video displays, and digital printers. As shown in 
FIG. 1, a color-co rTe.f|ipnJnnnfftrmptjoB l^pn be used to 
transform toe RGB color signals produced by the color 
image sensors 10 to form device color signals appropriate 
for the intended image display, and tone-scale transforma - 
tions 14 c an be used to produce the desired tone-scale 
characteristics between the scene and the display. To accom- 
plish this, it is necessary to know how the RGB color signals 
produced by the color sensors correspond to the perceived 
color values of the color stimulus in the scene as perceived 
by a human observer, and additionally how to produce the 
same perceived color on the display device. CIE tristimulus 
values (X, Y, and Z) are typically used to characterize the 
response of the human visual system to a color stimulus 



X-k x fI{\)lWxQ,)dX (2) 
Z-k z fr(k)R<k)%k)dk 

where x(k), y(X), and z(X) are the CIE color matching 
functions, and the constants kp ky> and k z are normalization 
factors. If the sensor spectral responsivities of the detectors 
are linear combinations of the CIE color matching functions 



5 J? (X)=a /u 4c(A.>+ < i ff ^<A,>M RZ 2<X) 



(3) 



it can be seen, by substituting equation (3) into equation (1), 
and converting to matrix notation, that the tristimulus values 
of a color stimulus the scene can be determined from the 
RGB color values by the relationship 



(4) 



where M c is a camera color-correction matrix given by 
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(5) 



Typically, the sensor spectral responsivities will not be linear 

40 combinations of the color-matching functions so that the 
matrix relationship given in Eq. (4) will not be strictly true, 
but in most cases a camera color-correction matrix can be 
computed using optimization techniques so that Eq. (4) 

45 represents a reasonable approximation. 

Once the scene tristimulus values are known, the next step 
is to determine the desired tristimulus values to be produced 
on the image display device. Typically, the illuminant for the 
original scene photographed by the camera will be different 

50 from the reference illuminant of the image display device. 
As a result, the tristimulus values of a white object in the 
scene will be different than the tristimulus values of an 
object that appears to be white on the image display device. 
However, a human observer will perceive them both to be 

55 white because of the chromatic adaptation capabilities of the 
human visual system. Therefore, in order to insure that a 
white object in the scene will appear to be white on the 
image display device, it is necessary to apply a chromatic 

^ adaptation transformation to the scene tristimulus values to 
determine the appropriate image display device tristimulus 
values. One such chromatic adaptation transformation that is 
frequently applied is the well-known Von-Kries transforma- 
tion. In the case of the Vbn-Kries transformation, as well as 

65 many other simple chromatic adaptation transformations, 
the chromatic adaptation transformation can be accom- 
plished using a simple matrix multiplication 
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(6) 



where X d , and Z d are the tristimulus values appropriate 
for the display device, and is the chromatic adaptation 
matrix which is a function of the reference whites for the 
scene and the display device. 

To produce the desired color on some display device, it is 
necessary to use a device model to determine what device 
control signals will produce the desired color. A video 
display is a common output device for a digital image. Video 
displays can typically be modeled quite accurately by a 
non-linear function describing the tone response of the video 
circuitry, followed by the application of a phosphor matrix 
that is related to the colors of the video phosphors. Typically, 
the non-linearity is approximately of the form 



'<-(-sr) 



(7) 20 



are the device color signals, y is a 
contrast parameter, and R c , G c , and B c are the linear inten- 
sity color signals. A phosphor matrix, M^, is used to deter- 
mine the device tristimulus values from the linear intensity 
control signals: 
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(8) 35 
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Therefore, to determine the device control signals necessary 
to produce a desired color, it is necessary to invert these 
operations by first applying the inverse phosphor matrix 



(9) 



followed by the inverse of the non-linear tone response 
function. 

To compute the device control signals from the camera 
color values the operations given in Eqs. (4), (6), and (9) 
should be applied sequentially. These equations represent 
sequential matrix operations that can therefore be combined 
into a single matrix operation 



(10) 



where M cc is a composite color-correction matrix given by 
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Due to implementation speed considerations, this composite 
color-correction matrix would typically be applied in the 
color-correction transformation 12 rather than the sequence 
of individual matrices. Note that if the channel dependent 
gain factors are set up so that neutral objects produce equal 



RGB color values, and if the output device has the usual 
characteristic that equal RGB device control signals produce 
a neutral image, then the composite color-correction matrix 
will have the feature that equal color values must produce 
equal device control signals. This is normally accomplished 
by requiring that the composite color-correction matrix have 
row-sums equal to 1.0. Following the color-correction 
matrix operation, tone-scale transformations 14 which typi- 
cally approximate the inverse of the device non-linearity. 

Usually, the color-correction operation is optimized for a 
particular assumed scene illuminant, as well as a particular 
output device. If the actual scene illuminant is different than 
the assumed scene illuminant, the color-correction operation 
will not produce the desired result. This is due to the fact that 
the different scene illuminant will have a different spectral 
content, and will therefore produce different RGB color 
values than the reference scene illuminant. The most notice- 
able result will usually be that neutral colors in the scene will 
not map to neutral colors on the display. This can easily be 
seen by noting that if the color-correction operation were set 
up so that the color-correction operation produces equal 
device control signals given equal RGB color values, then it 
will produce unequal control signals from non-equal RGB 
color values. As a result, the image will appear to have a 
color cast when the image is displayed. For example, if a 
camera is set up to produce equal RGB signals for daylight 
illumination, and the camera is then used to photograph a 
scene under tungsten illumination, the resulting image will 
generally appear to have a yellowish cast due to the fact that 
the spectral power distribution associated with tungsten 
illumination has a much lower level of blue content than the 
spectral power distribution associated with daylight. 

To account for this illuminant dependence, many systems 
have implemented a form of illuminant dependent gain 
control like that shown in FIG. 2. In this case, a digital 
camera has red, green and blue image sensors 20 to detect 
light imaged from a scene onto the sensors, and produces 
color signals R, G, and B. An illuminant determining process 
22 determines the illuminant dependent gain factors G^ G G , 
and G B that are applied to the color signals using multipliers 
24 to produce white-balanced color signals G Wy and Tl w . 
Note that if the color signals are not in a linear space the 
application of the gain factors will require a more complex 
operation than a simple multiplication. In such cases, the 
gain factors can be applied using functional operations, or 
alternatively fyy g y*pp1 yi.ng_T nn ^" i n -tables ( LUTsY Another 
approach would involve using linearization [unctions or 
ImearizatiojaXUTs to first convert the color signals to linear 
color signals where the gain factors could then be applied by 
a simple multiplication. The white-balanced color sign als 
are then _ color-corrected, usi up a_cx 3lojj QQrrection trans for- 
mation 26 and processed through tone-scale transformations 



"ancTB^ appro- 

priate for the intended image display device. The illuminant 
dependent gain factors are generally chosen so that the 
white-balanced color signals produced by photographing a 
neutral patch will be approximately equal. 

A number of types of illuminant determining processes 
have been used in the prior art. One such type of illuminant 
determining process is an optical color temperature detector 
disposed on the front of the camera. Another type of 
illuminant determining process includes the step of having a 
user identify a neutral patch in a captured image and 
computes the gain factors that would be necessary to equal- 
ize the color signals for that patch. Still another type of 
illuminant determining process includes having a user 
choose an illuminant type from a list of possible illuminant 
types. The list of possible illuminant types might include 
classes of iUurninants such as daylight, tungsten, flash, and 
fluorescent 
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The use of ilium in ant dependent gain factors will allow FIG. 2 shows a prior art color-correction process for a 
for the correction of the overall neutral balance of the image, digital camera which uses an illuminant detecting process to 
but there will still be residual color reproduction errors apply channeKdependent gain factors; 
associated with this simple illuminant correction approach. FIG. 3 shows a color-correction process according to the 
For example, although a gray patch in the image should be 5 present invention which uses an illuminant determining 
reproduced as gray, a red patch in the image may be process to determine an illuminant-dependent color- 
reproduced with an incorrect hue angle. This is due to the correction transformation; 

fact that the color-correction operation was designed for the rtG. 4 shows a color-correction process according for a 

assumed scene iUuminant, thus the camera model and the digital carnera according to present invention which includes 

chromatic adaptation correction may not be appropriate for 10 tone-scale transformations; 

the actual scene iUuminant. In order to fix the remaining mQS Sfl ^ fc e Je ^ ^^^^ 

color reproduction errors, it is not possible to simply adjust matfix ^^tnts as a Junction of daylight color tempera- 

the channel-mdepcndent gain factors Instead, the color- ^ according t0 ^ prescnt invention; 

correction transformation must by modified as well. , , , , - . c 

In U.S. Pat. No. 5,253,047 Machishima has recently is f G 6 shows ^ average color errors as a functoon of 

disclosed a method for using a color temperature detecting «*» ^mperature obtamed using the present inventon 

circuit to modify the matrix coefficients for the primary color ™™P*™1 to the average color errors obtamed when a fixed 

j ~[ r , . . r color-correction matrix is used; and 

separator used to perform a color-correction operation for a ' 

color video camera. The primary color separator is used to nG - 7 shows a color-correction process according for a 

compute the red, green and blue primary color signals from 20 <%i tal camera according to the present invention which 

the luminance/chrominance signals generated by the camera includes separate neutral-balance transformations, 

detector circuitry. In particular, two constants, referred to as DETAILED DESCRIPTION OF THE 

a and p, are varied in response to the signal from the color INVENTION 
temperature detecting circuit. This approach allows for some 

degree of compensation for variations in the scene 25 Referring now to FIG. 3, a generic system for practicing 

iUuminant, but it has the disadvantage that it does not permit the present invention will be described. A digital camera has 

for optimum correction because it does not allow for using multi-channel color image sensors 30 to detect light imaged 

all of the degrees-of-freedom available in the primary color from a scene onto the sensors. The digital camera may be a 

separator matrix. Further, it has the disadvantage that the digital still camera, or a digital video camera. In many 

color-correction operation can not be optimized for each 30 common digital cameras there are three types of color 

illuminant type so as to minimize average color errors. sensors having spectral sensitivities that are generally red, 

green, and blue. For purposes of illustration we will consider 

SUMMARY OF THE INVENTION a camera having sensors of this type, but it is clear that the 

It is an object of the present invention to color-correct method of the P resent invention can easily be extended to 
multichannel color image signals from a digital camera to 35 cameras having other types of color sensors such as cyan, 
account for variations in scene iUuminant including the steps magenta, and yellow, or cyan, magenta, yeUow, and green, 
of determining the scene illuminant, and determining an ^ multi-channel color image sensors produce multi- 
optimum color-correction transformation, adapted to pro- channel color signals shown in FIG. 3 as R, G, and B. An 
cess the color image signals, that minimizes color errors illuminant determining process 32 determines the iUuminant 
between an original scene and a reproduced image respon- 40 incident on the scene. A color-correction transform deter- 
sive to the scene iUuminant. fining process, 34 determines an optimum color-correction 

This object is achieved by a method for color-correcting transformation 36 responsive to the deterrnined Ulurninant. 

multichannel color image signals from a digital camera ^ optimum color-correction transformation 36 can then be 

, . . . , . . . ~ r . used to process the multi-channel color signals to form 

having multi-channel image sensors to account for vana- , . y . . . _ - , ~ ^. i.- . , 

- .„ . ..... Ik . f 45 device color signals R^, G A , and B rf . The multi-channel 

tions in scene iUuminant including the steps of: HJ . , ^ , • - * j- 1 

. , ... ... , output signals will generaUy be adapted for display on a 

a) determining the scene illuminant; and particular form of output device such as a video display. 

b) determining an optimum color-correction transforma- Alternatively, the multi-channel output signals may be 
tion in response to the scene iUuminant which transform device-independent color values. In this case, the device- 
minimizes color errors between an original scene and a SQ independent color values can then be transformed to device- 
reproduced image by adjusting three or more parameters. dependent color values for one or more output devices using 

ADVANTAGES device profiles that transform device-independent color val- 
ues into the appropriate device-dependent color values for a 

The present invention has the advantage that the color- particular output device. This last approach is used by 

correction process for digital camera images can be opti- 55 systems commonly known as color-management systems, 

mized as a function of the scene illuminant rather than A number of types of iUuminant determining processes 32 

making the compromises associated with optimizing the can be used. One such type of illuminant determining 

process for a single illuminant. process is an optical color temperature detector disposed on 

It has the additional advantage that the parameters of the the front of the camera. Another type of illuminant deter- 

color-correction process deterrnined for each iUuminant 60 mining process includes the steps of having a user identify 

produce the minimum average color errors given the degrees a neutral patch in a captured image, and characterizing the 

of freedom associated with the color-correction transforma- iUuminant based on the relative color signals for the neutral 

tion. patch. A variation on this process would include having the 

"RRIFF INSCRIPTION OF THF DRAWINGS ^ P 00 * 0 ^ 0 a ^ ecific neutral ^ part of a camera 

BRIEF DESCRIPTION OF THE DRAWINGS fi5 calibration procedure that would be performed each time the 

FIG. 1 shows a prior art color-correction process for a scene iUuminant changes. Still another type of iUuminant 

digital camera; determining process includes having a user choose an Ulu- 
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mutant type from a list of possible illuminant types. The list form of the color-correction transformation and the form of 

of possible illuminant types might include classes of illu- the combined color error metric, it may be possible to 

minants such as daylight, tungsten, flash, and fluorescent. determine the optimum color-correction transformation 

Another type of illuminant determining process would parameter values using simple linear least squares 

include analyzing the color signals for the captured digital 5 techniques, or it may be necessary to use non-linear opti- 

imagc to estimate the illuminant characteristics from the mization methods. Examples of non-linear optimization 

image information. Methods of this type might include methods include iterative Simplex optimization procedures, 

finding specular highlights in the scene, or analyzing high- simulated annealing techniques, and genetic algorithms, 

light to shadow transitions in the scene. Yet another type of The optimum color-correction transformation 36 can take 

illuminant detennining process includes characterizing the 10 m any forms. For example, it might include a simple matrix 

illuminant using a color measurement device such as a operation having adjustable matrix coefficients, it might, 

spectroradiometer, or a colorimeter. include a series of matrix and look-up table operations 

The process for determining the optimum color-correction having a set of adjustable parameter^ nr jt mifffrt include, ftn, 

transformation 34 includes determining a color-correction adjus ta bl e three-dimensional look-up table (3-D LLTO. In 

transformation that minimizes the color errors between the 15 yet" another embodiment, the optimum color-correction 

scene and the reproduced image given the determined scene transformation can be a set of one-dimensional functions 

illuminant. In one embodiment of the present invention, the applied to each of the multi-channel color signals. Because 

color-correction transformation may be implemented using of the fact that there are typically three color channels, there 

a color-correction matrix as was shown above in Eq. (10), will usually be a minimum of three parameters that can be 

where the matrix coefficients are adjustable. In this case, the 20 adjusted during the optimization process. For example, a 

process of determining the optimum color-correction trans- color-correction matrix will have nine adjustable matrix 

formation involves determining the adjustable matrix coef- coefficients, and a 3JLLUT will barejuiumfeer of adjustable 

ficient values that minimize the color errors between the param eters given by the number of lattice points in the 

..scene and the reproduced image. uu . 

Generally, the optimization process may involve using a 25 In one embodiment of the present invention, the color 

least-squares minimization technique to minimize the color image sensors 30, and the illuminant determining process 32 

errors for a specific set of scene color values. The set of are integral parts of the digital camera. The information 

scene colors used in the optimization process can include regarding the illuminant determined from the illuminant 

colors spanning the range of real world colors, as well as detector is stored as a part of a digital image data structure 

colors of particular importance such as skin -tones and 30 used to store the multi-channel color signals. The digital 

neutral colors. image data structure is then processed at a later time using 

The scene color values for the set of scene colors can be a d^ 1 * 1 ima S e processor adapted to receive the digital 

computed using well-known procedures given the spectral image from the digital camera. The digital image processor 

characteristics of the scene colors, and the determined ac mi ^ 1 be a computer workstation, or a specialized image 

illuminant. If the spectral characteristics of the scene illu- processing device. The digital image processor performs the 

minant have not been directly measured, they can be esti- color-correction transformation determining process 34 and 

mated using the information provided by the illuminant applies the optimum color-correction transformation 36 to 

determining process. For example, if only the estimated the multi-channel color signals to form multi-channel output 

color temperature of the illuminant has been measured, an signals. The information regarding the illuminant that is 

appropriate illuminant spectrum having the desired color can stored as a part of a digital image data structure might be raw 

be assumed. Alternatively, if the illuminant has been deter- sign* 1 values produced by the illuminant detector, color 

mined by having the user choose from a set of possible signals corresponding to a neutral object in the scene, an 

classes of ilium in ants, a representative illuminant spectrum estimated illuminant color temperature, a measured illumi- 

for that class of ilium in ants can be assumed. 45 nant spectra, or some other parameters) that characters the 

The color of the reproduced image associated with a given illuminant. 

color-correction transformation can either be characterized In another embodiment of the present invention the illu- 

by reproduced color values for a set of scene colors pro- minant determining process 32 is also performed in the 

duced using a particular output device, such as a video digital image processor. This step might include analyzing 

display or a printer, or can characterized by device- 50 the digital image data to estimate the scene ffluminant, or 

independent color values adapted for conversion to device- having a user select a scene illuminant from a set of possible 

dependent color values at a later time. The reproduced color scene ilium in ants. 

values can either be measured from an actual captured image In yet another embodiment of the present invention, the 
of a test target, or they can be determined from a knowledge optimum color-correction transformation determining pro- 
of the spectral sensitivities of the sensors. 55 cess 34 is performed and then the optimum color-correction 
The differences between the perceived scene color values transformation 36 are applied to the multi-channel color 
and the reproduced color values can be used to determine a signals to form multi-channel output signals are performed 
combined color error. One form of combined color error that by a digital image processing means integrated into the 
can be computed for a given color-correction transformation digital camera so that the output of the digital camera would 
is the root-mean-square (RMS) color error. One measure of 60 be me desired multi-channel output signals, 
color error that is frequently convenient to compute is the Those skilled in the art will appreciate that other alternate 
distance between the scene color and the reproduced color in embodiments are also possible. For example, the color- 
a device-independent color space such as the well-known correction transformation determining process 34 can be 
CIE tristimulus value color space. Other color spaces that performed in the camera and the resulting optimum color- 
can be used would include so-called uniform color spaces 65 correction transformation can be stored as a part of a digital 
such as CIELAB or CIELUV, or device-dependent color image data structure used to store the multi-channel color- 
spaces (such as video RGB code values). Depending on the signals. The stored optimum color-correction transformation 
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would include a set of parameters necessary to describe the that the RMS color error between the scene color values and 

optimum color-correction transformation. For example, the the reproduced color values is minimized. The RMS color 

parameters might include a set of color-correction matrix error AE* RAfS is simply given by 
coefficient values, pf t ~ rt n f ^ ^ " n 1lllT"» 

Frequently, the color=cQrrection transformation may 5 I (14) 

mcju de_^ pects of "to ne-s cale correction , as_weU _as„coIor- ^ I g 2 

correction ! bor" exlmftleTthe color-correction transformation ^ 5 "N 1=0 1 
may include 3* P I ,UT . Jbat_represents the combined 

o perations of ^ r.vertinf ? from the sensor color values to th e where N is the number of color patches. 

orimanL color values for^DartqciUar^de'o displaced 1Q It can be seen in FIGS. Sa-5c that the optimum matrix 

applying a desired tone-scale function. Many times it may be ^^1115 vary quite substantially across the range of color 

^nvenient to include a separate step for explicitly ^control- investigated, particularly for lower color tem- 

hng the image tone-scale Tins is shown in FIG. 4, where ^ ^ e ^lot errors for the set of color 

multi-cham,el color signals R G, and B are formed by image ^ fof ^ ^ color ^ orrectlon matrix 

sensors 40. An ilium in ant determining process 42 deter- . c r A , 4 , , 

4 . .„ • .j . ? a 1 15 at each color temperature are shown in FIG. 6. For 
mines the llluminant incident on the scene. A color- . *\ , . _ . , 
correction transform determining process 44 determines an comparison, a curve is also shown for the case where the 
optimum connection transformation 46 responsive to matnx wa * optimized for a 55 °°K 00 i lor temperature 
the determined illuminant. The optimum color-correction was ™* for ^ temperatures. It can be seen that the 
transformation 46 is used to process the multi-channel color color reproduction errors grow rapidly when the matrix is 
signals to form color-corrected color signals R<~, G 0 and ^ d at a color temperature it was not designed for. The color 
Be- The color-corrected color signals are then processed by errors obtained when the optimum color-correction matrix is 
tone-scale transformations 48 to form device color signals used for each color temperature arc substantially smaller 
R^, Gj, and B d . than those obtained for the 5500K color-correction matrix if 
For example, consider a digital still camera having red, ^ the color temperature departs significantly from 5500IC This 
green, and blue sensitive color image sensors to be pro- indicates that substantial improvements in the color repro- 
cessed using the configuration shown in FIG. 4. In this duction characteristics of the digital camera can be obtained 
example, the optimum color-correction transformation 46 by varying the matrix coefficients for the color-correction 
will be a 3x3 color-correction matrix to transform from the transformation accordingly. 

sensor color values to output primary color values appro- 30 In the above example, the optimum matrix coefficients 

priate for a typical video display. FIGS. 5a-Sc show the were computed with the assumption that the camera elec- 

matrix coefficient values associated with an optimum color- tronics are adjusted so that neutral scene colors produce 

correction transformation that are determined as a function equal RGB color values. This usually implies that gain 

of the color temperature of a blackbody illuminant: factors must be applied to the color values produced by each 

35 of the color channels. These gain factors will also typically 

oi3 It r ( 12 ) be a function of the illuminant. The application of the gain 

fl2 3 g factors can be included in the optimum color-correction 

a33 B transformation 46, or can be treated as a separate operation. 

FIG. 7 shows an embodiment of the present invention which 
The optimum matrix coefficient values were determined by 40 includes a separate neutral-balance correction step. A digital 
cascading the blackbody illuminant spectra having the camera has multi-channel color image sensors 70 to detect 
appropriate color temperature with a set of scene spectral light imaged from a scene onto the sensors. The multi- 
reflectance functions to form cascaded scene spectra. The channel color image sensors produce multi-channel color 
cascaded scene spectra were then used to determine the signals shown in FIG. 7 as R, G, and B. An illuminant 
camera RGB color signals given the camera spectral sensi- 45 determining process 72 determines the illuminant incident 
tivities using Eq. (1), and the perceived color values for each on the scene. A neutral-balance transform ctetermining pro- 
of the patches using Eq. (2). Reproduced color values can cess 74 determines the neutral-balance transform 76 neces- 
then determined by applying the optimum color-correction sary to produce neutral-balanced color signals R w G^ and 
transformation 46 and tone-scale transformations 48 to the B^ In this example, the neutral balance-transform 76 is 
camera RGB color signals to produce device color signals 50 shown as a set of simple gain factors G Ry G G , and G^ that 
Rj, Gj, and B^ for a CRT display. A model of the color are applied to each of the multi-channel color signals, 
reproduction characteristics for a CRT display can then be Alternatively, the neutral balance-transform can include 
used to determine the reproduced color values given the more complex operations such as one-dimensional look-up 
device color values. In this example, a set of patches from tables (1-D LUTs). A color-correction transform detenu in- 
aMacBeth Color Checker were used as scene colors, and the 55 ing process 77 determines an optimum color-correction 
color errors for each patch were determined by the well- transformation 78 responsive to the determined illuminant. 
known CIELAB AE* color-difference formula: The optimum color-correction transformation 78 can then be 

used to process the neutral-balanced color signals to form 

A£*,* (£,%• -l* £ ¥ + («% - a* & f + (f>% - b* £ ? < 13 > color-corrected color signals G c , and B c . The color- 

50 corrected color signals are then processed by tone-scale 

where L*^ a* ^ and b* rf are the CIELAB scene color values transformations 79 to produce device color signals R rf , G d9 

for the i* color patch, L* A , a* A> and b* A - are the CIELAB B </* 

reproduced color values for the I th color patch on the output The invention has been described with reference to a 

device, and AE* t - is the resulting color error for the i^ patch. preferred embodiment. However, it will be appreciated that 

A standard Simplex optimization procedure was used to 65 variations and modifications can be effected by a person of 

determine the values of the matrix coefficient values of Eq. ordinary skill in the art without departing from the scope of 

(12) for the optimum color-correction transformation such the invention. 
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Parts list 



10 color image sensors 

12 color-correction transformation 

14 tone-scale transformations 

20 image sensors 

22 flhuninant dete rminin g process 

24 multipliers 

26 color-correction transformation 

28 tone-scale transformations 

30 color image sensors 

32 Uluminant determining process 

34 color-correction transform determining process 

36 optimum color-correction transformation 

40 color image sensors 

42 Uluminant determining process 

44 color-correction transform determining process 

46 optimum color-correction transformation 

48 tone-scale transformations 

70 color image sensors 

72 illuminant determining process 

74 neutral-balance transform determining process 

76 neutral-balance transform 

77 color-correction transform determining process 

78 optimum color-correction transformation 

79 tone-scale transformations 



We claim: 

1.. A method for color-correcting multi-channel color 
image signals from a digital camera having multi-channel 
image sensors to account for variations in scene illuminant 
comprising the steps of: 

a) determining the scene illuminant; and 

b) determining an optimum illuminant-dependent color- 
correction transformation based on the determined 
scene illuminant which transformation minimizes color 
errors between an original scene and a reproduced 
image when applied to the multi-channel color signals 
to produce multi-channel color output signals, each of 
such multi-channel color output signals being depen- 
dent upon more than one of the multi-channel color 
image signals. 

2. The method of claim 1 wherein the scene illuminant is 
determined using an optical color temperature detector on 
the digital camera. 

3. The method of claim 1 wherein the scene illuminant is 
determined from the relative color signals produced by 
photographing a neutral object in the scene. 

4. The method of claim 1 wherein the scene illuminant is 
determined by analyzing the color image data for the scene. 

5. The method of claim 1 wherein the scene illuminant is 
determined by having a user select the scene illuminant from 
a list of scene ilium in ants. 

6. The method of claim 1 wherein the digital camera is a 
digital still camera. 

7. The method of claim 1 wherein the digital camera is a 
digital video camera. 

8. A method for color-correcting multi-channel color 
image signals from a digital camera having multi-channel 
image sensors to account for variations in scene illuminant 
comprising the steps of: 

a) determining the scene Uluminant; and 

b) determining an optimum color-correction transforma- 
tion in response to the scene illuminant which trans- 
formation minimizes color errors between an original 
scene and a reproduced image when applied to the. 
multi-channel color image signals to produce multi- 
channel color output signals, each of such multi- 
channel color output signals being dependent upon 
more than one of the multi-channel color image signals 
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wherein the optimum color-correction transformation 
determining step includes combining the color errors 
for a set of typical scene colors and determining the 
optimum color-correction transformation that mini- 
mizes the combined error. 
9. The method of claim 8 wherein the combined color 
error is the root mean square AE* value for the set of typical 
scene colors, the root mean square AE* value being given by 
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N 
Ni-0 



15 



where N is the number of typical scene colors, i 
particular typical scene color, and 



is the CIELAB color difference between the scene color 
values for the i* typical scene color specified by L*^, 
and b*^-, and the corresponding color of the reproduced 
image specified by L* A> a* A , and b%. 

10. A method for color-correcting multi-channel color 
image signals from a digital camera having multi-channel 
image sensors to account for variations in scene illuminant 
comprising the steps of: 

a) determining the scene illuminant; and 

b) determining an optimum color-correction transforma- 
tion in response to the scene illuminant which trans- 
formation minimiz es color errors between an original 
scene and a reproduced image when applied to the 
multi-channel color image signals to produce multi- 
channel color output signals, each of such multi- 

. channel color output signals being dependent upon 
more than one of the multi-channel color image signals 
wherein the color-correction transformation is a color- 
correction matrix having adjustable matrix coefficients. 

11. The method of claim 10 wherein the optimum color- 
correction transformation is determined by determining the 
adjustable matrix coefficients that minimize the color errors 
between the original scene and the reproduced image. 

12. The method of claim U wherein the matrix coeffi- 
cients that minimize the color errors between the original 
scene and the reproduced image are determined by mini- 
mizing the color errors for a set of typical scene colors. 

13. The method of claim 12 wherein the optimum color- 
correction transformation determing step includes minimiz- 
ing color errors by minimizing the root mean square AE* 
value for the set of typical scene colors, the root mean square 
AE* value being given by 



50 



N 
Ni=0 



where N is the number of typical scene colors, i 
particular typical scene color, and 



is the CIELAB color difference between the scene color 
values for the i^ typical scene color specified by L*„, a* 5l -, 
and b*^-, and the corresponding color of the reproduced 
image specified by L* A , a*^, and b* A . 

14. The method of claim 1 wherein the color-correction 
transformation is an adjustable three-dimensional look-up 
table that stores output color values for a lattice of input 
color values. 
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15. The method of claim 1 wherein information describ- 
ing the determined scene ilium in ant is stored as part a data 
structure used to store the color image signals. 

16. The method of claim 15 wherein the information 
describing the determined scene illuminant is an ilium inant s 
color temperature. 

17. The method of claim 15 wherein the information 
describing the determined scene illuminant is an illuminant 
spectrum. 

18. The method of claim 15 wherein the information 10 
describing the determined scene illuminant is an identifier 
for one of a set of possible scene illuminants. 

19. The method of claim 1 wherein information describ- 
ing the optimum color-correction transformation is stored as 
part a data structure used to store the color image signals, is 

20. The method of claim 19 wherein the information 
describing the optimum color-correction transformation 
includes matrix coefficient values for a color-correction 
matrix. 

21. The method of claim 1 further including the step of 20 
applying the optimum color-correction transformation to the 
color image signals in the digital camera. 

22. The method of claim 1 further including the step of 
applying the optimum color-correction transformation to the 
color image signals in a digital image processor adapted to 25 
receive the color image signals from the digital camera. 

23. The method of claim 1 wherein the color-correction 
transformation transforms the color image signals from the 
digital camera to color image signals adapted for display on 

a video display device. 30 

24. The method of claim 1 wherein the color-correction 
transformation transforms the color image signals from the 
digital camera to device-independent color image signals. 

25. The method of claim 1 wherein the multi-channel 
image sensors are red, green, and blue image sensors. 35 

26. A method for color-correcting multi-channel color 
image signals from a digital camera having multi-channel 
image sensors to account for variations in scene illuminant 
comprising the steps of: 

a) determining the scene illuminant; 40 

b) classifying the scene illuminant into one of a set of 
possible scene illuminants; and 

b) selecting an optimum illuminant-dependent color- 
correction transformation based on the classified scene 45 
illuminant from a set of color-correction 
transformations, each transformation having been pre- 
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determined to minimize color errors between an origi- 
nal scene and a reproduced image for a particular 
classified scene illuminant when applied to the multi- 
channel color image signals to produce multi-channel 
color output signals, each of such multi-channel color 
output signals being dependent upon more than one of 
the multi-channel color image signals. 

27. A method for color-correcting multi-channel color 
image signals from a digital camera having multi-channel 
image sensors to account for variations in scene illuminant 
comprising the steps of: 

a) determining the scene illuminant; 

b) determining channel-dependent neutral-balance trans- 
formations based on the determined scene illuminant to 
be applied to the multi-channel color image signals for 
form neutral-balanced color image signals, the neutral- 
balance transformations being adapted to produce 
equal signal levels for scene colors that are neutral; and 

c) determining an optimum itiuminant-dependent color- 
correction transformation based on the determined 
scene illuminant which transformation minimizes color 
errors between an original scene and a reproduced 
image when applied to the multi-channel color image 
signals to produce multi-channel color output signals, 
each of such multi-channel color output signals being 
dependent upon more than one of the multi-channel 
color image signals. 

28. An apparatus for color-correcting multi-channel color 
image signals from a digital camera having multi-channel 
image sensors to account for variations in scene illuminant, 
comprising: 

a) means for determining the scene illuminant; and 

b) means for determining an optimum illuminant- 
dependent color-correction transformation based on the 
determined scene illuminant which transformation 
minimizes color errors between an original scene and a 
reproduced image when applied to the multi-channel 
color image signals to produce multi-channel color 
output signals, each of such multi-channel color output 
signals being dependent upon more than one of the 
multi-channel color image signals. 

29. The invention of claim 28 further including a digital 
camera for producing multi-channel color signals. 

* * * * * 



